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(54) Method and apparatus for coding/decoding items of subtitling data 



(57) Subtitling can be based on either pixel data or 
on character data. Character data allow very efficient 
encoding, but from character strings alone, subtitling 
can not be converted into a graphical representation to 
be overlaid over video. The intended character set, font 
and e.g. font size, must either be coded explicitly within 
the subtitling bitstream or an implicit assumption must 
be made about them. In pixel-based subtitling, subtitling 
frames are conveyed directly in the form of graphical 
representations by describing them as (typically rectan- 
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gular) regions of pixel values on the AV screen, at the 
cost of considerably increased bandwidth for the subti- 
tling data. According to the invention, a font memory is 
used that allows an efficient realisation of pixel-based 
subtitle lettering, because the glyphs need only be trans- 
mitted once and thereafter are referenced by relatively 
compact character references during the AV event. 
Thereby the invention combines the advantages of pure 
pixel-based and pure-character-based subtitling 
schemes, while mostly avoiding their respective short- 
comings. 
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Description 

[0001] The invention relates to a method and to an 
apparatus for coding/decoding items of subtitling data, 
in particular subtitling and graphics for Blu-ray disc op- 5 
tical storage and recording. 

Background 

[0002] In the area of subtitling for pre-recorded Audio- 
Visual (AV) material, conflicting requirements exist: On 
one hand, subtitling data should be efficiently encoded, 
especially if a whole bouquet of subtitling services is to 
be provided for any given AV material. In this case, at 
least on average, very few bits are available per subti- 
tling character. 

On the other hand, professional content owners want to 
have full control over the appearance of subtitling char- 
acters on screen, additionally they want to have at their 
command a rich set of special display effects from sim- 
ple fading all through to genuine animations. Such high 
degree of design freedom and command normally is 
feasible only with high or very high subtitling bandwidth. 
[0003] Two main approaches exist in today's state of 
the art for subtitling pre-recorded AV data signals with 
separate subtitling information: Subtitling can be based 
on either pixel data or on character data. In both cases, 
subtitling schemes comprise a general framework, 
which for instance deals with the synchronisation of sub- 
titling elements along the AV time axis. 
In the character-based subtitling approach, e.g. in the 
TELETEXT system (see ETSI: ETS 300 706 Enhanced 
Teletext specification, May 1997) for European analog 
or digital TV, strings are described by sequences of let- 
ter codes, e.g. 

[0004] ASCI I (see ISO/IEC 8859: American Standard 
Code for Information Interchange - ASCII) or UNICODE 
(see ISO/IEC 10646: Information technology -- Univer- 
sal Multiple-Octet Coded Character Set (UCS)), which 
intrinsically allows for a very efficient encoding. But from 
character strings alone, subtitling can not be converted 
into a graphical representation to be overlaid over video. 
For this, the intended character set, font and some font 
parameters, most notably the font size, must either be 
coded explicitly within the subtitling bitstream or an im- 
plicit assumption must be made about them within a suit- 
ably defined subtitling context. Also, any subtitling in this 
approach is confined to what can be expressed with the 
letters and symbols of the specific font or fonts in use. 
The DVB Subtitling specification (see ETSI: ETS 300 
743 Digital Video Broadcasting (DVB); Subtitling sys- 
tems, Sep 1997, and EP-A-0 745 307: Van derMeeret 
al, Subtitling transmission system), with its object types 
of 'basic object, character" or 'composite object, string 
of character 1 , constitutes another state-of-the-art exam- 
ple of character-based subtitling. 
[0005] In the pixel-based subtitling approach, subti- 
tling frames are conveyed directly in the form of graph- 



ical representations by describing them as (typically rec- 
tangular) regions of pixel values on the AV screen. 
Whenever and wherever anything is meant to be visible 
in the subtitling plane superimposed onto video, its pixel 
values must be encoded and provided in the subtitling 
bitstream, together with appropriate synchronisation in- 
fo. Obviously removing any limitations inherent with 3rd 
party defined fonts, the pixel-based approach carries 
the penalty of a considerably increased bandwidth for 
the proper subtitling data. Examples of pixel-based sub- 
titling schemes can be found in DVD's 'Sub-picture' con- 
cept (see DVD Forum: DVD Specifications for Read-On- 
ly Disc/ Part 3 Video Specifications/ Version 1.0 August 
1996) as well as in the 'bitmap object* concept of DVB 
Subtitling (see ETS 300 743 and EP-A-0 745 307 men- 
tioned above). 

Invention 

[0006] A problem to be solved by the invention is to 
combine the efficient encoding of character-based sub- 
titling with full control over the appearance of subtitling 
characters as is feasible with pixel-based subtitling, 
without significantly increasing the data amount re- 
quired for transferring the necessary information. This 
problem is solved by the methods disclosed in claims 1 
and 7. An apparatus that utilises the method of claim 1 
is disclosed in claim 4. 

[0007] The invention is based on a pixel-based subti- 
tling scheme. This subtitling system includes several 
components which allow to include font support into an 
otherwise pixel-based subtitling scheme. This font sup- 
port includes: 

a.1) A structure for Font Describing Data for effi- 
ciently describing a set of font characters in pixel 
data form; 

a.2) A structure for Font Identification Data to 
uniquely identify a predefined font to be used; 
a.3) A concept of having a font memory as a part of 
the overall memory area, wherein that font memory 
is dedicated to hold the font characters, and is not 
directly visible in the AV output; 
a. 4) A structure for Character Referencing Data for 
efficiently referencing individual font characters 
from amongst the font or fonts stored in the font 
memory. 

[0008] Font Describing Data as well as Character Ref- 
erencing Data are transmitted or stored alongside AV 
data, whereby that transmission or storage has either 
the format of a nearly inseparable mix or uses complete- 
ly separate transmission channels or storage locations, 
or is a mix of both. 

At decoder side the Font Describing Data cause a set 
of arbitrary character glyphs (graphical representation 
of a character) or other graphics building blocks to be 
loaded into the font memory. The number and design of 
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character glyphs to be used in each individual case is 
completely under the control of the content provider. 
[0009] According to the invention, the Font Describing 
Data consist of one or more character parameter parts 
each comprising character parameter sets of one ore 
more characters in the font and one or more character 
pixel data parts each comprising the pixel data of one 
or more characters in the font. The pixel data of a char- 
acter are represented as a character array, i.e. as a rec- 
tangular array of pixel values, the array having a width 
and a height specific to the character. Each one of said 
character parameter sets includes any combination of: 

c.1) The width of the character array; 
c.2) The height of the character array; 
c.3) The start address of the pixel data of the char- 
acter relative to the character pixel data part con- 
taining it; c.4) A horizontal offset between the 
boundaries of the array and a character reference 
point; 

c.5) A vertical offset between the boundaries and 
the character reference point; 
c.6) A horizontal increment describing the horizon- 
tal distance between the character and those char- 
acters to either precede or succeed it. 

[0010] The inventive use of a font memory provides 
an efficient realisation of pixel-based subtitle lettering, 
because the glyphs need only be transmitted once and 
thereafter are referenced by relatively compact charac- 
ter references during the AV event. 
On the other hand, because glyphs are effectively pro- 
vided in pixel-based form, the appearance of subtitling 
is entirely put under content provider's control, and all 
problems of font identification, font selection, font para- 
metrisation and character rendering, which normally 
come with character- based schemes, are avoided ad- 
vantageously. 

[0011] In this way, the invention actually combines the 
advantages of pure pixel-based and pure-character- 
based subtitling schemes, while mostly avoiding their 
respective shortcomings. 

[0012] In principle, the inventive method is suited for 
decoding items of subtitling data, including the steps: 



writing said pixel data into said display memory as 
designated by said items of Character Positioning 
Information. 

s [0013] In principle the inventive apparatus is suited for 
decoding items of subtitling data, said apparatus includ- 
ing: 

means for retrieving items of Character Referenc- 
10 ing Data that are related to corresponding parts of 
a video or audio-visual data signal, which data items 
describe sequences of characters as well as infor- 
mation about where in pictures of said data signal 
and/or when and/or how to make the referenced 
15 characters visible using a display memory; 
means for: 

deriving from said items of Character Referenc- 
ing Data items of Character Selecting Informa- 
20 tion and Character Positioning Information; 

reading pixel data of said referenced charac- 
ters as designated by said items of Character 
Selection Information from a font memory; 
writing said pixel data into said display memory 
25 as designated by said items of Character Posi- 

tioning Information. 

[0014] Advantageous additional embodiments of the 
invention are disclosed in the respective dependent 
30 claims. 

Drawings 

[0015] Exemplary embodiments of the invention are 
35 described with reference to the accompanying draw- 
ings, which show in: 



»f the inventive subtitling sys- 

structure for embedding a 
DVD-ST 'object_data_seg- 



[0016] As illustrated in Fig. 1 , the Font Describing Da- 
ta 102 as well as the Character Referencing Data 103 
are transferred, stored or recorded together with related 
AV data 101, whereby the transmission or storage can 
be anything between a nearly inseparable mix and the 
use of completely separate transmission channels or 
storage locations. 

At decoder side, as shown in Fig. 2, a subtitling stream 
201 passes through data separation means 202, which 
in turn provides Character Referencing Data 203 and 
Font Describing Data 204. By passing a font describing 
data processing means 205, the Font Describing Data 
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Fig. 1 Inventive data si 
Fig. 2 Block diagram o 
40 tern; 

Fig. 3 Example data 
'font_id' into a 
ment'. 

45 Exemplary embodiments 

retrieving items of Character Referencing Data that 
are related to corresponding parts of a video or au- 
dio-visual data signal, which data items describe 
sequences of characters as well as information 
about where in pictures of said data signal and/or so 
when and/or how to make the referenced charac- 
ters visible using a display memory; 
deriving from said items of Character Referencing 
Data items of Character Selecting Information and 
Character Positioning Information; 55 
reading pixel data of said referenced characters as 
designated by said items of Character Selection In- 
formation from a font memory; 
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204 cause a set of arbitrary character glyphs or other 
graphics building blocks to be loaded into a font memory 
208. 

Advantageously, the number and design of character 
glyphs to be used in each individual use case is com- 5 
pletely under content provider's control. 
Optionally, to a font thus described and loaded into font 
memory 208, the above-mentioned Font Identification 
Data can be associated. 

The Character Referencing Data 203 cause character 
referencing data processing means 206 to copy individ- 
ual subsets of the set of character glyphs denoted Char- 
acter Describing Data 209 from font memory 208 into a 
display memory 207, which can be a part of the overall 
system memory. The content of display memory 207 
gets overlaid onto video and hence becomes a visible 
subtitle. 

[0017] Optionally, the Character Referencing Data 
can contain references to the Font Identification Data, 
thus allowing a subtitling decoder to decide whether a 
font required for rendering a specific subtitling stream 
must still be loaded into font memory 208, or is already 
available for immediate use. 

[0018] Possible uses and modes of operation of the 
proposed subtitling system can include, but are not lim- 
ited to, one of: 

b. 1 ) Pre-loading at least one font for use throughout 
a long AV program; 

b.2) Use of fonts containing more than one variant 
for at least one of the letters, the use of which in- 
cludes, but is not limited to, subpixel-accurate letter 
positioning or emphasis (bold/italic) support; 
b.3) Loading font subsets for parts of AV material 
(e.g. movie chapters) in cases where sparse sub- 
sets of big fonts are used, like e.g. Asian fonts. 

[0019] For the further structure of the Font Describing 
Data, several variants of specific embodiment are pro- 
posed as follows. 

In a first variant, if the font is a proportional font where 
individual characters have variable width, all the char- 
acter arrays are horizontally padded to be nominally of 
equal width, and the resulting padded character arrays 
are vertically concatenated into a font array. The font 
array is then line-scanned in conventional way to form 
a single character pixel data part. 
Inanothervariant, all character arrays a re vertically pad- 
ded to be nominally of equal height, and the resulting 
padded character arrays are horizontally concatenated 
into a font array. The font array is then line-scanned in 
conventional way into a single character pixel data part. 
For both above variants, the single character pixel data 
part is preceded by a single character parameter part 
comprising the character parameter sets of ail charac- 
ters in the font. 

In another variant, the Font Describing Data are gener- 
ated by alternately concatenating the character param- 
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eter sets and the character arrays, for all characters in 
the font. 

[0020] In another variant, the Font Describing Data 
are generated by first concatenating all the character 
parameter sets into a single character parameter part, 
and appending to that part a single character pixel data 
part comprising all the character arrays. 
In another variant, which may optionally extend all 
above variants, a UNICODE (see ISO/IEC 10646: Infor- 
mation technology - Universal Multiple-Octet Coded 
Character Set (UCS)) code is associated to some or all 
of the characters of the font, and the UNICODE code is 
inserted and included at an identifiable position within 
that part of the Font Describing Data which is associated 
with the character in question. 

[0021] In another variant, which may optionally ex- 
tend all above variants, a non-repetitive character iden- 
tifier is associated to every character of the font, and the 
identifier is inserted and included at an identifiable po- 
sition within that part of the Font Describing Data which 
is associated with the character in question. 
[0022] In all above variants, the Font Describing Data 
can either be 

d.1) directly transmitted using one codeword per 
data item, or they can be 

d.2) compressed by runlength coding, or they can 
be 

d.3) compressed by other methods for lossless 
compression such as the 'zlib' method used in PNG 
(see W3C recommendation, PNG (Portable Net- 
work Graphics) Specification, Version 1.0, 1996, ht- 
tpillwww. w3.org/TRIREC-png.pdf). 

[0023] For the structure of the Font Identification Da- 
ta, several variants of specific embodiment are pro- 
posed as follows. In a first variant, the Font Identification 
Data structure is embodied as a Tontjd' as defined in 
the 'Portable Font Resource' (PFR) system (see Bit- 
stream Inc.: TrueDoc PFR Specification, httpM www. bit- 
stream. com/ pfrspecf index, html) . 
[0024] In another variant, the Font Identification Data 
structure in the form of a PFR 'font_id' is embodied into 
the above-mentioned DVB subtitling system, using a da- 
ta structure as illustrated in Fig. 3. 
In another variant, the Font Identification Data structure 
is embodied as a "Universally Unique Identifier" as de- 
fined in (UUID in: ISO/IEC 11578:1996, Information 
technology - Open Systems Interconnection - Remote 
Procedure Call (RPC)). 

[0025] In the context of the invention, the Character 
Referencing Data consist of a sequence of one or more 
character reference groups each accompanied by 
group positioning data, and each character reference 
group consists of a sequence of one or more character 
references each accompanied by character positioning 
data. 

[0026] The group positioning data can preferably be 
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embodied as one of: 

e.1) Absolute horizontal and vertical coordinates of 
a group reference point relative to the origin of the 
video image; 5 
e.2) Relative horizontal and vertical coordinates of 
the group reference point relative to the group ref- 
erence point of the previous character reference 
group; 

e. 3) Relative horizontal and vertical coordinates rel- 10 
ative to any other prescribed reference point. 

[0027] The character references can preferably be 
embodied as one of: 

15 

f. 1) Character indexes referring to the implicit posi- 
tion of the designated character within the Font De- 
scribing Data; f.2) Any kind of unambiguous char- 
acter identifiers; 

f.3) ASCII codes if they have been unambiguously 20 
assigned to the characters; 

f. 4) UNICODE codes if they have been unambigu- 
ously assigned to the characters. 

[0028] The character positioning data can preferably 25 
be embodied as one of: 

g. 1) An automatic advance needing no additional 
individual character positioning data, the advance 
being deductible from the position of the character 30 
reference point of the previous character and from 

the horizontal increment of the character in ques- 
tion; 

g.2) An automatic advance with character position 
offset data, where for the horizontal as well as for 35 
the vertical position of the character a first value de- 
duced from the position of the character reference 
point of the previous character and from the hori- 
zontal increment of the character in question is add- 
ed with a second value which is individually de- *o 
scribed in the character positioning data; 
g.3) Relative character positioning data applied rel- 
ative to the character reference point of the previous 
character; g.4) Absolute character positioning data 
applied relative to the video image origin. *s 



Claims 

1 . Method for decoding items of subtitling data, char- so 
acterised by the steps: 

retrieving (202) items of Character Referencing 
Data (103, 203) that are related to correspond- 
ing parts of a video or audio-visual data signal 55 
(101), which data items (1 03, 203) describe se- 
quences of characters as well as information 
about where in pictures of said data signal and/ 



or when and/or how to make the referenced 
characters visible using a display memory' 

(207) ; 

deriving (206) from said items of Character Ref- 
erencing Data (103, 203) items of Character 
Selecting Information and Character Position- 
ing Information; 

reading (206) pixel data of said referenced 
characters as designated by said items of Char- 
acter Selection Information from a font memory 

(208) ; 

writing (206) said pixel data into said display 
memory (207) as designated by said items of 
Character Positioning Information. 

2. Method according to claim 1 , wherein the following 
steps are carried out before retrieving (202) said 
items of Character Referencing Data (103, 203): 

retrieving (202) items of Font Describing Data 
(102, 204) related to corresponding ones of 
said items of Character Referencing Data (1 03, 
203); 

writing (205) said items of Font Describing Data 
into said font memory (208). 

3. Method according to claim 1 or 2, wherein, after re- 
trieving said items of Character Referencing Data 
(103, 203), the following steps are carried out: 

checking whether or not said pixel data of said 
referenced characters are already stored in 
said font memory (208) ; 
if not true, retrieving (202) such items of Font 
Describing Data (102, 204) which contain said 
referenced characters; 

writing said items of Font Describing Data into 
said font memory (208). 

4. Apparatus fordecoding items of subtitling data, said 
apparatus including: 

means (202) for retrieving items of Character 
Referencing Data (1 03, 203) that are related to 
corresponding parts of a video or audio-visual 
data signal (101), which data items (103, 203) 
describe sequences of characters as well as in- 
formation about where in pictures of said data 
signal and/or when and/or how to make the ref- 
erenced characters visible using a display 
memory (207); 
means (206) for: 

deriving from said items of Character Ref- 
erencing Data (103, 203) items of Charac- 
ter Selecting Information and Character 
Positioning Information; 
reading pixel data of said referenced char- 
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acters as designated by said items of Char- 
acter Selection Information from a font 
memory (208); 

writing said pixel data into said display 8. 
memory (207) as designated by said items 5 
of Character Positioning Information. 



items of Font Describing Data into said font 
memory. 

A data carrier containing a video or audio-visual da- 
ta signal (101) and related subtitling data that are 
encoded using a method according to claim 7. 



5. Apparatus according to claim 4, wherein said 
means (202) for retrieving, before retrieving said 
items of Character Referencing Data (1 03, 203), re- 10 
trieve items of Font Describing Data (102, 204) re- 
lated to corresponding ones of said items of Char- 
acter Referencing Data (103, 203), said apparatus 
further including: 

15 

means (205) for writing said items of Font De- 
scribing Data into said font memory (208). 



6. Apparatus according to claim 4 or 5, further includ- 
ing means for checking, after retrieving said items 20 
of Character Referencing Data (1 03, 203), whether 

or not said pixel data of said referenced characters 
are already stored in said font memory (208), 
wherein, if not true, such items of Font Describing 
Data (1 02, 204) are retrieved that contain said ref- 25 
erenced characters, and are written into said font 
memory (208). 

7. Method for encoding subtitling data, characterised 

by the step: 30 



attaching to a video or audio-visual data signal 
(101) related subtitling data including items of 
Character Referencing Data (103, 203) and 
items of Font Describing Data (102, 204), 35 
whereby said items of Character Referencing 
Data (1 03, 203) describe sequences of charac- 
ters as well as information about where in pic- 
tures of said data signal and/or when and/or 
how to make the referenced characters visible 40 
using a display memory, said items of Charac- 
ter Referencing Data including items of Char- 
acter Selecting Information and Character Po- 
sitioning Information, wherein said items of 
Character Selection Information can be used in 45 
a subtitle decoder for reading pixel data of said 
referenced characters from a font memory and 
said items of Character Positioning Information 
can be used in said subtitle decoder for writing 
said pixel data into said display memory, so 
and whereby said items of Font Describing Da- 
ta (1 02, 204) can be written in said subtitle de- 
coder into said font memory for checking 
whether or not said pixel data of said refer- 
enced characters are already stored in said font 55 
memory and, if not true, retrieving such items 
of Font Describing Data (102, 204) which con- 
tain said referenced characters and writing said 
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ob]ect_data_segment ( ) { 

if (object_coding_method =='01') { 

number_of_codes 8 uimsbf 

if ( number_of_codes ==0) { 

for (i = 1; ; i ++) { 

font ID [ i ] 8 uimsbf 

if (font ID [i] == 0 ) break; 

} 

number_of_codes 8 uimsbf 

} 

for (i = 1; i <= number_of_codes; i++) { 

character_code 16 bslbf 

} 

} 
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